Acceleration of Primal–Dual Methods by Preconditioning and Simple Subproblem Procedures

Primal–dual hybrid gradient (PDHG) and alternating direction method of multipliers (ADMM) are popular first-order optimization methods. They easy to implement have diverse applications. As methods, however, they sensitive problem conditions can struggle reach the desired accuracy. To improve their performance, researchers proposed techniques such as diagonal preconditioning inexact subproblems. This paper realizes additional speedup about one order magnitude. Specifically, we choose general (non-diagonal) preconditioners that much more effective at reducing total numbers PDHG/ADMM iterations than ones. Although subproblems may lose closed-form solutions, show it suffices solve each subproblem approximately with a few proximal-gradient or epochs proximal block-coordinate descent, which simple steps. Global convergence this approach is proved when inner fixed. Our opens choices maintains both low per-iteration cost global convergence. Consequently, on several typical applications primal–dual obtain 4–95 $$\times $$ over existing state-of-the-art.